The present application relates generally to an improved data processing apparatus and method and more specifically to mechanisms for backing up metadata in a data storage system.
In known data storage systems, data structures are used to describe where user data may be located and how it may be retrieved. Such data structures are known as metadata. Usually, such metadata are mostly stored on disk, with only some parts being held in memory. For performance considerations, updates made to the metadata stored in memory are not immediately performed in respect of the metadata stored on disk, but are rather cached and periodically flushed to disk from memory. Whilst this approach may facilitate improved performance of such data storage systems, it may contribute to metadata inconsistency in the case of a system failure, which may result in some of the metadata remaining unfinished from memory to disk, in order to address this issue, a journal is implemented in order to serve as a record of metadata stored on memory and updates performed in respect thereof. Additionally, and in order to recover from system failures that may cause corruption of the metadata held on disk, such as, for example, disk failure, data storage systems may be configured so that a backup of the metadata stored on disk is performed by collecting snapshots of such metadata at specifiable and regular intervals. Such a backup is sometimes referred to as a metadata snapshot and/or a point-in-time metadata backup.
The occurrence of a system failure may cause a loss of any update information on metadata that is stored only on memory of the storage system and/or also the flushing of updated metadata from memory to disk to be stalled. Accordingly, metadata stored on disk of the storage system is rendered stale and it should, when a system reboot is performed, preferably be manipulated to correspond in update status to the most updated version of metadata stored on memory just before the occurrence of the system failure. Such updating of the stale metadata on disk may be performed by: accessing the journal comprising the most recently recorded updates made to the metadata on memory; comparing the journal entries with back-up snapshots of the corresponding metadata stored on disk; identifying stale metadata on the disk where a discrepancy is recorded between metadata in a given backup, metadata snapshot and corresponding metadata of more current, updated status in a given journal entry; applying changes to such stale metadata thereby to align its status with the more current, updated metadata corresponding thereto in the given journal entry of the journal. This process is referred to as replaying the journal.
There are some challenges associated with known data storage systems. One such challenge is associated with any further updates/changes to the metadata stored on memory being quiesced when a backup of metadata stored on disk is performed. This is done by stopping/blocking any host writes whilst snapshots of the metadata stored on disk are collected. Once such a backup has been completed, host writes may be resumed and any new updates/changes to the metadata written to the memory are recorded in the journal entries of a new, updated journal. Because host writes are stopped/blocked when such a backup is performed, this may contribute to a degradation of the performance characteristics of data storage systems in which such a backup process is used. This issue may be addressed by facilitating host writes to be continued while the backup is performed and storing any updated metadata in an intermediate and relatively large cache before being transferred to memory on completion of the backup. In this solution, the opportunity of continued host writes to be performed during metadata backup may need to be balanced with an increased risk of loss of data on memory, and an increased management and/or storage capacity overhead.
A further challenge is associated with the fact that, whilst the increased data storage capacity of relatively modern storage systems is desirable, this translates to metadata structures of increased size. Some problems associated with such metadata of increased size are that a relatively longer time may be needed both for taking snapshots during backup of such metadata and quiescing host writes when such a backup is performed. These issues may be addressed by breaking down the larger metadata structure(s) into multiple, smaller metadata structures, which may be backed up respectively over a relatively shorter period of time during which host writes would be stopped. However, consideration would need to be given to increased management and/or processing overheads that may be incurred in respect of such multiple, smaller metadata structures.
Accordingly, it is a challenge to mitigate some of the drawbacks associated with known data storage systems.